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ACT is an independent, not-for-profit organization whose mission is helping 
people achieve education and workplace success. And it has been our pleasure in 
having had this opportunity to work with the National Council for Community 
and Education Partnerships (NCCEP) in its work of developing research-based 
college access programs and supporting the implementation of proven educational 
strategies. The following report is the culmination of our work. 

In 2005/2006, ACT and NCCEP collaborated to collect data for evaluating 
student gains in academic achievement, course planning behavior, and 
commitment to college plans. An elemental study was created to evaluate the 
effectiveness of the federal GEAR UP (Gaining Early Awareness and Readiness 
for Undergraduate Programs) program, whose goal is to prepare students for 
college. 

Comparisons were measured between GEAR UP and Non-GEAR UP schools at 
grade levels 8 and 10, using the ACT EPAS™ (Educational Planning and 
Assessment System). The study found that GEAR UP programs make a difference 
when compared to Non-GEAR UP schools regarding academic readiness and 
college intent. We found that GEAR UP students were more likely to be on track 
as college-ready, more likely to be taking the necessary core curriculum, and more 
likely to have plans for college by 10th grade. We also were able to make 
suggestions and specific recommendations to schools regarding best practices. 

What we have learned with this comprehensive, longitudinal study is important to 
continued improvement in GEAR UP schools and how they help their students. 
Since many students in GEAR UP schools come with multiple challenges to their 
academic success, intervention programs become critical to raising the skills and 
expectations of these students. They previously may not have considered finishing 
high school, let alone further education. As we evaluate the various intervention 
programs of individual schools, we develop a better understanding of how best to 
assist students in becoming successful in their academic careers in both high 
school and as they continue on. 

In studying this process, we have also learned how to evaluate GEAR UP 
programs and in what areas we need further study and follow up. And while there 
is continued room for improvement in helping students prepare for their future, 
this report signifies that we are on the right path. 

I would like to thank all of those whose hard work and continuous efforts went 
into creating this study and report. I look forward to continued partnership 
between ACT, NCCEP, and GEAR UP, and to a brighter future for our country’s 
young people. 

Cynthia B. Schmeiser 

President and COO, Education Division, ACT, Inc. 
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For the last eight years, the National Council for Community and Education 
Partnerships (NCCEP) has been working to increase educational expectations, 
preparation, and success for low-income students and their communities across 
the country. We do this by supporting and strengthening research-based and 
practice-proven educational intervention strategies, like the federal GEAR UP 
program, for which we serve as the national intermediary organization and 
technical assistance provider. We believe that GEAR UP is one of the most 
promising educational strategies for decreasing the academic achievement gap 
and increasing this country’s global competitiveness. 

As the GEAR UP program has grown in size and reputation, serving millions 
of students in its short history and producing positive data such as drastically 
increased high school graduation rates, NCCEP has continually worked to refine 
the program for maximum effectiveness. We emphasize as part of our 
organization’s operating principles that sound research and evaluation should 
be at the core of both the program’s growth in the policy arena as well as in 
impacting the program’s refinement at the state and local levels. 

It is for that reason that we engaged in a research collaboration with ACT to 
investigate some of the impact that GEAR UP was having beyond the data 
currently collected. In partnering with ACT, we choose an organization that is not 
only a leader in educational research as it relates to academic achievement and 
rigor but also one that understands the vital links between research, effective 
practice, and policy development. It is a tremendous asset to the GEAR UP and 
broader college access communities that ACT has joined this evaluative effort. 

Our study sought to ask the broadest research question possible about GEAR UP— 
do students benefit by having GEAR UP in their school? The results in this report 
indicate that the answer is yes, and speak to the promise of GEAR UP. Despite the 
study limitations, the results indicate that GEAR UP is having a positive effect on 
students. Because of the study limitations, we have every reason to believe that the 
effect of GEAR UP is even greater than indicated. It is especially encouraging to 
see these kinds of results for GEAR UP over such a short period of time. More 
work needs to be done to refine GEAR UP and better understand its impact, but 
the results of this study, coupled with other data being collected nationally, allow 
us to move ahead with the knowledge that GEAR UP is working. 

NCCEP views this report as another step in the still nascent research and 
evaluation being conducted on GEAR UP. We will continue to seek avenues to 
advance and accelerate efforts in this direction. I encourage others to do so as 
well. We look forward to nurturing our partnership with ACT so that together we 
can continue to advance what we know about the positive impact of GEAR UP 
and college access programming. 



Hector Garza 

President, National Council for Community and Education Partnerships (NCCEP) 



Executive Summary 

In this report, we compare changes in academic readiness and college 
intent for a sample of students from GEAR UP schools to a comparable 
sample from Non-GEAR UP schools. We utilize assessment data from 
ACT’s EXPLORE® and PLAN® programs to measure students’ academic 
readiness and college intent at grade 8 and grade 10. Therefore, we are 
able to measure the degree that GEAR UP affects change between these 
two grades. Since GEAR UP programs begin no later than grade 7 and 
continue on past grade 10, we are only able to measure GEAR UP’s effect 
for a portion of the intervention period. Further, the best indicator of 
GEAR UP’s success will be whether college enrollment and retention rates 
improve for students from GEAR UP schools. Still, growth between 
grade 8 and grade 10 is crucial for college preparedness, as many 
students set their future educational course during this time period. 

Our analyses suggest that the students from GEAR UP schools are 
slightly better than their Non-GEAR UP counterparts with respect to 
changes in academic readiness and college intent from grade 8 to 
grade 10. 

Highlighting the findings: 

▼ Students from GEAR UP schools had slightly greater changes in 
overall academic performance from grade 8 to grade 10. Relative to 
the Non-GEAR UP comparison group, students in the GEAR UP group 
gained 0.16 more composite scale score points, on average, for one 
of the cohorts studied. For the other cohort, there was no significant 
difference in change in overall academic performance. 

▼ Students from GEAR UP schools were slightly more likely to be on 
track to be college-ready in English and Reading. Relative to the 
Non-GEAR UP comparison group, the odds of being college-ready 
were 16% and 27% higher for the GEAR UP group in English and 
Reading, respectively, for one of the cohorts studied. For the other 
cohort, there was no significant difference in the odds of being 
college-ready in English or Reading. 

▼ Students from GEAR UP schools were slightly more likely to take the 
core high school curriculum and have plans for college at grade 10. 
These findings applied to just one cohort studied; for the other, there 
was no significant difference in taking the core high school curriculum 
or having plans for college. 



In the Discussion section of this report, we talk about future 
evaluation efforts to ameliorate the limitations of the current study design. 
Specifically, our analysis would have been more likely to detect effects of 
the GEAR UP program if our data set had included information on which 
students received which interventions, and the duration and intensity of 
each student’s intervention. Since interventions will vary across GEAR UP 
schools, it stands to reason that the outcomes affected will also vary 
across schools. For example, some programs may be more likely to affect 
changes in academic performance, while others are more likely to affect 
changes in knowledge of the college admissions process. By combining 
data across GEAR UP programs, we might lose the ability to detect these 
program-specific effects. 

As detailed in the report, we selected our comparison group of schools 
based on their similarity to the GEAR UP schools. But, we acknowledge 
that any attempt to find a comparison group for GEAR UP schools will be 
imperfect. Since GEAR UP schools have the highest poverty levels, any 
comparison of GEAR UP schools to Non-GEAR UP schools will be flawed. 
In our analyses, we attempted to overcome the discrepancy in school 
poverty level by controlling for school’s poverty level through regression 
modeling. 

This study’s findings suggest that the GEAR UP program has an effect on 
changes between grade 8 and grade 10; the study also sheds light on 
how GEAR UP programs can be properly evaluated. In the Discussion 
section, we give specific recommendations to schools and GEAR UP 
evaluators. We stress the need to track student-level intervention data as 
well as the need to track long-term student outcomes such as college 
enrollment, retention, and degree completion. We also recommend 
longitudinal assessment systems, such as ACT’s EPAS (Educational 
Planning and Assessment System) program for measuring student and 
school-level growth from the middle school years all the way through 
high school. 
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Introduction 



Background 

The Gaining Early Awareness and Readiness for Undergraduate Program 
(GEAR UP) is designed to provide assistance to low income students. 

The program provides discretionary grants for the purpose of increasing 
the readiness of low income students to attend and succeed in 
postsecondary education. The grants are up to six years in length and 
provide services to a cohort of students who are then followed from 
middle school through high school. 

A reasonable question to ask is how a state or partnership can show that 
its GEAR UP program is having the desired effect. In this report, we build 
upon past work by PPSS (2003) and Terenzini, et al. (2005) in an attempt 
to measure GEAR UP’s effects. We report on analyses using EXPLORE 
and PLAN data for the evaluation of GEAR UP programs. EXPLORE, an 
assessment typically given in 8th grade, and PLAN, an assessment 
typically given in 10th grade, are ideally suited for measuring the level of 
change between 8th and 10th grade. Both tests are intended to measure 
skills required for postsecondary success, and both have indicators of 
plans for postsecondary education. 

Research Questions 

The National Council for Community and Education Partnerships (NCCEP) 
provided electronic files to ACT identifying schools that participated in the 
GEAR UP program for three academic years (2002-2003, 2003-2004, 
and 2004-2005). For each academic year, staff at ACT determined which 
schools participated in ACT’s EXPLORE program, which schools 
participated in ACT’s PLAN program, and which schools participated in 
both. We report on outcomes for two cohorts of students from GEAR-UP 
schools: students that took EXPLORE in grade 8 during the 2002-2003 
academic year and later took PLAN in grade 10 during the 2004-2005 
academic year, and a subsequent cohort of students that took EXPLORE 
in grade 8 during the 2003-2004 academic year and later took PLAN in 
grade 10 during the 2005-2006 academic year. 

Using the EXPLORE and PLAN data, the GEAR UP program can be 
evaluated with respect to changes in academic readiness and changes in 
educational plans. As detailed later in this report, the GEAR UP schools 
can be compared to similar Non-GEAR UP schools that also participated 
in EXPLORE and PLAN. Since our sample data include baseline 
measures (EXPLORE data) and follow-up measures (PLAN data), we can 
address research questions related to changes in outcomes that are 
attributable to the GEAR UP program. The research questions we address 
are all concerned with college preparedness and college intent. By 
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utilizing EXPLORE and PLAN data, we can study the level of change that 
occurs for specific cohorts of students between 8th and 10th grade. The 
research questions we address include: 

1. Is there a difference between GEAR UP and Non-GEAR UP groups in 
changes in overall academic achievement from 8th to 10th grade? 

2. Is there a difference between GEAR UP and Non-GEAR UP groups in 
changes in percentage of students who are on track for being 
prepared for college in four subject areas (English, mathematics, 
reading, and science)? 

3. Is there a difference between GEAR UP and Non-GEAR UP groups in 
changes in percentage of students who are planning to take the core 
high school curriculum? 

4. Is there a difference between GEAR UP and Non-GEAR UP groups in 
changes in percentage of students planning to go to college? 



Method 

Selection of Matching Schools 

Data were drawn from ACT’s EXPLORE and PLAN history files 
corresponding to the academic years of interest (i.e., EXPLORE data 
from 2002-2003 and 2003-2004 and PLAN data from 2004-2005 and 
2005-2006). For each GEAR-UP school that participated in EXPLORE 
and PLAN, we selected a Non-GEAR UP school that also participated in 
EXPLORE and PLAN to serve as a control. Each Non-GEAR UP school 
was matched to a particular GEAR UP school based on combinations of 
the following school-level characteristics: 

▼ Mean EXPLORE Composite score 

▼ Grade level EXPLORE was administered 

▼ Control (public vs. private) 

▼ Enrollment size 

▼ Number of EXPLORE-tested students 

T Metropolitan area (urban, rural, or suburban) 

As an example of the matching process, for the first cohort (EXPLORE- 
tested students of 2002-2003), 120 GEAR UP schools participated in the 
EXPLORE program and 119 matched to Non-GEAR UP schools on the 
school characteristics listed above. Some GEAR UP schools matched 
with more than one Non-GEAR UP school, providing us with a pool of 
Non-GEAR UP schools to select from. The most similarly matching school 
was selected from this pool. The main criteria for matching schools was to 
keep the difference in mean EXPLORE Composite score within one point. 
Without eliminating imperfectly matching pairs of schools, we tried to 
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maximize the number of matching variables. Before finalizing the 
matching process, we attempted to match on school’s poverty level and 
school’s state, in addition to the variables listed above. However, 
matching on these variables resulted in too many schools for which no 
adequate match was found. Rather than eliminating non-matching 
schools, we decided to eliminate poverty level and state as matching 
criteria. The matching process was executed in the same fashion for both 
cohorts. Table 1 gives the resulting sample sizes of school pairs and 
students for the two cohorts (labeled as Cohorts 1 and 2). 



Table 1: Sample Sizes for Cohorts Studied 



Cohort 

Number 




Number of 
Matching 
School Pairs 


Number of Students 


Year and Assessments 


GEAR UP 


Non-GEAR UP 


1 


2002/2003 EXPLORE & 
2004/2005 PLAN 


119 


6,270 


5,808 


2 


2003/2004 EXPLORE & 
2005/2006 PLAN 


136 


6,707 


5,791 



The Appendix includes tables that compare the GEAR UP and 
Non-GEAR UP groups on school and student-level characteristics. 

Table 24 compares the GEAR UP and Non-GEAR UP schools for both 
cohorts. All GEAR UP and Non-GEAR UP schools were public. On 
average, GEAR UP and Non-GEAR UP paired schools have the same 
aggregate achievement in grade 8 (mean EXPLORE Composite scores). 
The two groups of schools are similar on the other matching 
characteristics (the number of students tested, enrollment size, and 
locale). For both cohorts, there is a discrepancy in the poverty levels of 
the GEAR UP and Non-GEAR UP schools. There are more GEAR UP 
schools at the highest level of poverty (25% or more) and more 
Non-GEAR UP schools at the lowest level of poverty. 

Table 25 in the Appendix compares the two groups with respect to 
student-level characteristics. The overall student-level sample sizes are 
slightly higher for the GEAR UP group. Both groups include a higher 
concentration of female students. Relative to the Non-GEAR UP group, 
the GEAR UP group has a higher percentage of Hispanic students and 
lower percentages of White and African-American students. The 
percentage of students whose parents did not complete high school is 
greater for the GEAR UP group for both cohorts. For example, 15.3% of 
the mothers in the GEAR UP group did not complete high school as 
compared to 1 1 .9% in the Non-GEAR group for Cohort 1 . This 
discrepancy is similar for Cohort 2: 14.3% as compared to 1 1 .5%. 

Results of the matching process are satisfactory, though imperfect, at the 
school and student levels for both cohorts studied. Comparing outcomes 
of students in GEAR UP and Non-GEAR UP schools is now meaningful 
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since we have eliminated most of the differences in school environments. 
That is, students within the Non-GEAR UP schools can be thought of as 
the control group that did not experience the GEAR UP program, yet 
experienced similar school environments. Therefore, differences in 
outcomes will not be attributed to the school environment but can be 
attributed to the GEAR UP program. 

Outcomes Studied 

The investigation focused on the level of college readiness and college 
intent in GEAR UP schools and their matched counterparts. Baseline 
(EXPLORE) and follow-up (PLAN) measures were available for all 
students, allowing us to compare changes from grade 8 to grade 10. The 
outcome variables we studied include the following: 

▼ Changes in EXPLORE and PLAN Composite scores. 

▼ Changes in meeting EXPLORE and PLAN College Readiness 
Benchmarks for each subject area. 

T Changes in meeting EXPLORE and PLAN College Readiness 
Standards for each subject area. 

▼ Plans for taking core high school curriculum at grade 10. 

▼ Changes in plans for college from grade 8 to grade 10. 

In the following paragraphs, we describe these outcomes in greater 
detail. 

Changes in EXPLORE and PLAN Composite scores. The EXPLORE 
Composite score is the mean of four multiple choice tests in English, 
mathematics, reading, and science. Each test, and the composite, range 
from 1 to 25. The tests measure students’ curriculum-related knowledge 
and cognitive skills important for future education and careers (ACT, 

2001). Similarly, the PLAN Composite score is the mean of four multiple 
choice tests in the same four subject areas. The PLAN Composite scores 
range from 1 to 32 and measures student development in the same way 
as the EXPLORE Composite score, with the main difference being that the 
two tests focus on skills attained at different times in the students’ 
educational experience (ACT, 1999). Therefore, a student’s change in 
composite score (from EXPLORE to PLAN) can be used as a measure of 
academic growth. Since GEAR UP programs typically begin in 7th grade, 
we hope to see greater academic growth for the GEAR UP group 
between 8th and 10th grade. 

Changes in meeting EXPLORE and PLAN College Readiness Benchmarks. 

Change in composite score is a measure of overall academic growth, but 
does not directly address college readiness. The second research 
question asks: Is there a difference between GEAR UP and 
Non-GEAR UP groups in changes in percentage of students who are on 
track for being prepared for college? ACT provides two different ways of 
establishing college readiness for EXPLORE and PLAN test scores: the 
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College Readiness Benchmarks (cf . , Allen and Sconing, 2005) and the 
College Readiness Standards™. The College Readiness Benchmarks 
have been established for EXPLORE, PLAN, and the ACT®. For the ACT, 
they measure whether a student has the knowledge required to succeed 
in an entry-level credit-bearing college course. For both EXPLORE and 
PLAN, the benchmarks measure whether a student is on track to meet the 
ACT benchmark. The scale score cutoffs for meeting the EXPLORE and 
PLAN benchmarks, respectively, in the four subject areas are: English 
(13, 15), Mathematics (17, 19), Reading (15, 17), and Science (20, 21). If 
GEAR UP programs are effective, we should see students increase their 
levels of readiness, as measured by the benchmarks. 

Changes in meeting EXPLORE and PLAN College Readiness Standards. 

The College Readiness Standards are a categorization of the EXPLORE, 
PLAN, and ACT score ranges into discrete levels. For each level, the 
standards delineate what students at that level can do (for further details 
see www.act.org/standard/index.html). If GEAR UP programs are 
effective, we should see students increase their levels of readiness, as 
measured by their standards level. 

Plans for taking the core high school curriculum at grade 10. Recent policy 
reports have suggested that taking rigorous courses in high school is a 
key to being prepared for college and work (Adelman, 1999; ACT, 2005). 
Taking the core high school curriculum is an indicator of a student’s plans 
and readiness for college and the workplace. When students take the 
PLAN assessment in 10th grade, they are asked to provide information 
about the high school courses they are taking or planning to take in the 
future. Using this information, we can determine which students are taking 
(or have plans for taking) the core high school curriculum. We consider 
the core high school curriculum to be four years of English and three 
years each of mathematics, science, and social studies. If GEAR UP 
programs are leading students to take more courses aligned with college 
expectations, we should see more GEAR UP students taking the core 
high school curriculum by 10th grade. 

Changes in plans for college from grade 8 to grade 10. When students take 
the EXPLORE assessment, they are asked what their future educational 
plans are. They can mark one of the following options: not planning to 
complete high school, no education or other training planned, job training 
offered through military service, apprenticeship or other on-the-job 
training, vocational or technical school, two-year community college or 
junior college, four-year college or university, undecided about future 
educational plans, or other. When students take the PLAN assessment, 
they are again asked what their future educational plans are. We 
classified students as having college plans at grade 8 and grade 10 if 
their response was “two-year community college or junior college” or 
“four-year college or university.” If their response was “undecided about 
future educational plans” or “other,” we did not classify them as having or 
not having college plans. By considering this outcome variable, we can 
measure the extent to which GEAR UP students change their college 
plans from 8th to 10th grade. 
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Statistical Analyses 

For each research question, we compared the GEAR UP and 
Non-GEAR UP groups on the relevant outcome of interest. For each 
outcome, our analysis entailed two steps: 1) descriptive analysis 
comparing means or frequency distributions of the outcomes for the 
two groups, and 2) regression analysis to test the statistical significance of 
the group differences, while controlling for baseline measures. In order to 
detect possible cohort differences, these two steps are performed 
separately for the two cohorts. 

Since the first outcome (change in Composite score from EXPLORE to 
PLAN) is an interval-scaled variable, a linear regression model will be 
used. All other outcomes can be dichotomized and logistic regression 
models (see Agresti, pp. 84-90) will be used. For both types of models, 
we will account for variation across school pairs by using hierarchical 
modeling (see Raudenbush & Bryk, 2002). The hierarchical linear 
regression models will be fit using SAS PROC MIXED software (SAS, 

2004) while the hierarchical logistic regression models will be fit using the 
SAS GLIMMIX macro (Wolfinger & O’Connell, 1993). For each regression 
model, we report estimates and 95% confidence intervals for the model’s 
parameters. For the linear regression model, the GEAR UP estimates 
represent the average change in the outcome attributable to the GEAR 
UP program. For the logistic regression models, we report the odds ratio 
(and 95% confidence interval) associated with the GEAR UP program. 

The odds ratio, calculated as exp((3) where [3 is a logistic regression 
coefficient, is a measure often used to describe the strength of 
association between a predictor and a dichotomous outcome. For 
example, if the odds ratio associated with the GEAR UP program is 1.20, 
then we estimate that the odds of “success” are 1 .20 times higher for the 
students from GEAR UP schools relative to students from Non-GEAR UP 
schools. Alternatively, we could interpret the odds ratio as the percentage 
increase in the odds of success. For example, an odds ratio of 1.20 could 
be interpreted as “the odds of success are 20% higher for students from 
GEAR UP schools.” An odds ratio of one implies that no relationship was 
detected between the predictor and dichotomous outcome. Therefore, if 
the 95% confidence interval for the odds ratio contains one, then we do 
not consider the odds ratio statistically significant. 

From our comparison of the school-level characteristics of GEAR UP and 
Non-GEAR UP schools (see Table 24), we know that GEAR UP schools 
have higher poverty levels than their matched Non-GEAR UP 
counterparts. Since a school’s poverty level often has a pronounced effect 
on students’ academic achievement, comparing outcomes for the two 
groups without adjusting for poverty level is somewhat misleading. 
Therefore, we included school poverty level as a covariate in our 
regression models in order to control for this discrepancy. To more 
precisely measure poverty level, we used a continuous indicator of each 
school’s poverty level: proportion of students eligible for free or reduced 
lunch. The categorical version of this variable, obtained through Market 
Data Retrieval (www.schooldata.com) is summarized for the two cohorts 
in Table 24. We obtained the continuous version of this variable from the 
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National Center for Education Statistics’ Common Core of Data (Sable, 
Thomas, & Shen, 2006). Unfortunately, the continuous poverty level 
indicator was not available for all GEAR UP and Non-GEAR UP schools. 
For Cohort 1 , the variable was available for 89 of the 119 school pairs; for 
Cohort 2, the variable was available for 100 of the 136 school pairs. 
Therefore, the sample of data used in the regression analyses is smaller 
than the overall sample. 



Results 



Changes in Composite Scores 

The first research question asked: Is there a difference between GEAR UP 
and Non-GEAR UP groups in changes in overall academic achievement 
from 8th to 10th grade? This question entails a comparison of the 
two groups with respect to changes in composite scores from EXPLORE 
to PLAN. EXPLORE and PLAN Composite score means are given in 
Figure 1 for Cohort 1 and Figure 2 for Cohort 2. 

Figure 1: Mean Composite Scores for Cohort 1 
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Figure 2: Mean Composite Scores for Cohort 2 
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Figures 1 and 2 suggest that the GEAR UP and Non-GEAR UP groups 
increased their composite scores at about the same level. When 
calculating the PLAN means, we only considered the students that had 
also taken EXPLORE. Therefore, the PLAN and EXPLORE means represent 
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identical samples of students and there is no selection bias. As a result of 
the matching process described earlier, EXPLORE means were about the 
same for the two groups. Comparing the PLAN means to the EXPLORE 
means, we see typical increases in composite scores of 1 .84 for GEAR UP 
and 1 .79 for Non-GEAR UP students in Cohort 1 , and typical increases of 
1.90 for GEAR UP and 2.07 for Non-GEAR UP schools in Cohort 2. So, 
from inspecting the means, it appears that the gains are very comparable 
for the two groups with the GEAR UP group having slightly higher gains in 
Cohort 1, but the Non-GEAR UP group gaining more in Cohort 2. 

In addition to the simple comparison of means, we modeled the change 
in composite score (PLAN Composite - Explore Composite) as a function 
of group (GEAR UP vs. Non-GEAR UP) and school’s poverty level. The 
results of the regression models are summarized in Table 2 for Cohort 1 
and Table 3 for Cohort 2. 



Table 2: Modeling Change in Composite Score-Cohort 1 



Predictor 


Estimate 
(95% C.I.) 


Interpretation 


GEAR UP 


0.16 

(0.07, 0.24) 


After controlling for school’s poverty level, students 
in the GEAR UP group gained 0.16 composite 
score points more than their Non-GEAR UP 
counterparts. 


School’s poverty level 


-1.25 

(-1.59, -0.90) 


On average, students at higher poverty schools 
see smaller increases in their composite scores. 



Table 3: Modeling Change in Composite Score-Cohort 2 



Predictor 


Estimate 
(95% C.l.) 


Interpretation 


GEAR UP 


-0.05 

(-0.14, 0.04) 


After controlling for school’s poverty level, students 
in the GEAR UP group gained slightly fewer 
composite score points than their Non-GEAR UP 
counterparts. 


School’s poverty level 


-1.05 

(-1.40, -0.69) 


On average, students at higher poverty schools 
see smaller increases in their composite scores. 



From Table 2 we see evidence that students in the GEAR UP group in 
Cohort 1 had greater changes in their composite scores relative to their 
Non-GEAR UP counterparts. After adjusting for school’s poverty level, 
students at GEAR UP schools in Cohort 1 gained 0.16 Composite score 
points more than their Non-GEAR UP counterparts. We generally consider 
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differences that are 0.50 or more scale points (on EXPLORE, PLAN, and 
ACT score scales) as practically important. Since this difference is 
smaller, it would not be considered practically important using this 
criterion. However, this finding is statistically significant and points toward 
a small positive effect of GEAR UP on changes in academic achievement. 
This finding was not consistent across the two cohorts: for Cohort 2, there 
was no apparent effect of GEAR UP on change in composite score. The 
estimate of the parameter representing the GEAR UP effect was -0.05, 
but was not significantly different than zero since the 95% confidence 
interval (-0.14, 0.04) contained zero (see Table 3). 

College Readiness Benchmarks 

Comparing changes in composite score for GEAR UP and Non-GEAR UP 
schools provided a measure of GEAR UP’s effect on overall academic 
achievement, but does not directly address GEAR UP’s effect on college 
readiness in specific subject areas. The second research question asks: 

Is there a difference between GEAR UP and Non-GEAR UP groups in 
changes in percentage of students who are on track for being prepared 
for college in four subject areas (English, mathematics, reading, and 
science)? To help answer this question, we can compare the two groups 
with respect to the percentage of students meeting the College 
Readiness Benchmarks. 

Results for meeting and not meeting the College Readiness Benchmarks 
for the two cohorts are given in Tables 4 and 5, respectively. For each 
subject area, these tables list the number of students who met the 
EXPLORE (grade 8) benchmark. Then, for each EXPLORE met 
benchmark/did not meet benchmark group, we list the number of 
students who met the corresponding PLAN (grade 10) benchmark. These 
statistics are presented separately for the GEAR UP and Non-GEAR UP 
schools. As an example, looking at Table 4, 2,619 GEAR UP students did 
not meet the EXPLORE English benchmark at grade 8. Of these students, 
924 met the PLAN English benchmark at grade 10. Therefore, 35% of the 
GEAR UP students who were not on track to be ready for an entry-level 
credit-bearing English course subsequently improved to the point that 
they were on track to be ready in grade 10. This result can be compared 
with students in the Non-GEAR UP group who showed the same rate of 
improvement. From inspecting Tables 4 and 5, there does not appear to 
be any major differences in the GEAR UP and Non-GEAR UP groups with 
respect to meeting the PLAN benchmarks. It is apparent that students 
that met the benchmarks in grade 8 are much more likely to meet the 
benchmarks in grade 10. It is also apparent that many students are 
slipping in their mathematics college readiness between grade 8 and 
grade 10. Only about half of the students that met the mathematics 
benchmark in grade 8 go on to meet the benchmark in grade 10. The 
results appear to be quite similar for the two cohorts. 
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Table 4: Rates of Meeting Benchmarks from 
EXPLORE to PLAN-Cohort 1 




Table 5: Rates of Meeting Benchmarks from 
EXPLORE to PLAN-Cohort 2 




We fit logistic regression models for meeting PLAN benchmarks 
that adjust for the discrepancy in poverty level. In addition to group 
(GEAR UP versus Non-GEAR UP) and poverty level, we also included 
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whether the student met the EXPLORE benchmark as a predictor. This 
model was fit for each PLAN benchmark (English, mathematics, reading, 
and science) and for both cohorts. The results of these models are 
summarized in Table 6 for Cohort 1 and Table 7 for Cohort 2. 

Table 6: Model for Meeting PLAN Benchmarks- Cohort 1 



Subject 

Area 


Predictor 


Odds Ratio 
(95% C.I.) 


Interpretation 




GEAR UP 


1.16 

(1.04, 1.29) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP group were more likely to meet the PLAN 
English benchmark. 


English 


School’s 
poverty level 


0.53 

(0.34, 0.80) 


Students from schools with higher poverty levels 
were less likely to meet the PLAN English benchmark. 




Met 

EXPLORE 

benchmark 


11.19 

(10.07, 12.44) 


The probability of meeting the PLAN English 
benchmark was much higher for students who met 
the EXPLORE benchmark. 




GEAR UP 


1.11 

(0.96, 1.27) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP group were slightly more likely to meet the 
PLAN Math benchmark. 


Math 


School’s 
poverty level 


0.27 

(0.15, 0.46) 


Students from schools with higher poverty levels 
were less likely to meet the PLAN Math benchmark. 




Met 

EXPLORE 

benchmark 


16.05 

(14.00, 18.42) 


The probability of meeting the PLAN Math 
benchmark was much higher for students who met 
the EXPLORE benchmark. 




GEAR UP 


1.27 

(1.14, 1.41) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP group were more likely to meet the PLAN 
Reading benchmark. 


Reading 


School’s 
poverty level 


0.22 

(0.15, 0.34) 


Students from schools with higher poverty levels 
were less likely to meet the PLAN Reading 
benchmark. 




Met 

EXPLORE 

benchmark 


9.00 

(8.09, 10.00) 


The probability of meeting the PLAN Reading 
benchmark was much higher for students who met 
the EXPLORE benchmark. 




GEAR UP 


1.02 

(0.88, 1.17) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP group had about the same probability of 
meeting the PLAN Science benchmark. 


Science 


School’s 
poverty level 


0.20 

(0.12, 0.34) 


Students from schools with higher poverty levels 
were less likely to meet the PLAN Science benchmark. 




Met 

EXPLORE 

benchmark 


18.23 

(15.07, 22.05) 


The probability of meeting the PLAN Science 
benchmark was much higher for students who met 
the EXPLORE benchmark. 
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Table 7: Model for Meeting PLAN Benchmarks- Cohort 2 



Subject 

Area 


Predictor 


Odds Ratio 
(95% C.l.) 


Interpretation 




GEAR UP 


1.02 

(0.91, 1.15) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP sample have about the same probability of 
meeting the PLAN English benchmark. 


English 


School’s 
poverty level 


0.36 

(0.24, 0.55) 


Students from schools with higher poverty levels 
are less likely to meet the PLAN English benchmark. 




Met 

EXPLORE 

benchmark 


10.36 

(9.32, 11.52) 


The probability of meeting the PLAN English 
benchmark is much higher for students who met the 
EXPLORE benchmark. 




GEAR UP 


1.14 

(0.97, 1.33) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP group are slightly more likely to meet the 
PLAN Math benchmark. 


Math 


School’s 
poverty level 


0.12 

(0.07, 0.22) 


Students from schools with higher poverty levels 
are less likely to meet the PLAN Math benchmark. 




Met 

EXPLORE 

benchmark 


17.67 

(15.38, 20.31) 


The probability of meeting the PLAN Math 
benchmark is much higher for students who met the 
EXPLORE benchmark. 




GEAR UP 


1.09 

(0.98, 1.23) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP sample are slightly more likely to meet the 
PLAN Reading benchmark. 


Reading 


School’s 
poverty level 


0.25 

(0.17, 0.37) 


Students from schools with higher poverty levels 
are less likely to meet the PLAN Reading benchmark. 




Met 

EXPLORE 

benchmark 


9.10 

(8.17, 10.13) 


The probability of meeting the PLAN Reading 
benchmark is much higher for students who met the 
EXPLORE benchmark. 




GEAR UP 


1.19 

(1.01, 1.42) 


After adjusting for poverty level and whether or not 
the EXPLORE benchmark was met, students in the 
GEAR UP sample are more likely to meet the PLAN 
Science benchmark. 


Science 


School’s 
poverty level 


0.08 

(0.04, 0.15) 


Students from schools with higher poverty levels 
are less likely to meet the PLAN Science benchmark. 




Met 

EXPLORE 

benchmark 


19.09 

(15.72, 23.17) 


The probability of meeting the PLAN Science 
benchmark is much higher for students who met 
the EXPLORE benchmark. 



From Tables 6 and 7, we see consistent evidence that higher 
school poverty levels lead to lower probabilities of meeting the PLAN 
benchmarks. We also see that meeting the EXPLORE benchmarks greatly 
increases the likelihood of meeting PLAN benchmarks. We see 
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inconsistent evidence of GEAR UP’s effect on meeting the PLAN 
benchmarks. For Cohort 1, students in the GEAR UP group are more 
likely to meet the PLAN English benchmark, once poverty level has been 
adjusted for. Since the odds ratio is greater than one (1.16) and the 
confidence interval (1.04, 1.29) does not contain one, there is evidence 
that the odds of meeting the PLAN English benchmark was greater for 
students in the GEAR UP group. For Cohort 2, the GEAR UP effect is not 
statistically significant since the odds ratio’s confidence interval (0.91, 

1.15) contains one. This suggests that students have about the same 
probability of meeting the PLAN benchmark, regardless of group. For the 
PLAN math benchmark, the positive GEAR UP effect is consistent across 
cohorts but again lacks statistical significance. For Cohort 1, students in 
the GEAR UP group have an 1 1% increase in the odds of meeting the 
PLAN math benchmark; for Cohort 2, students in the GEAR UP group 
have a 14% increase. For the PLAN reading benchmark, the GEAR UP 
effect is stronger for Cohort 1 : students in the GEAR UP group have a 
27% increase in the odds of meeting the PLAN reading benchmark; for 
Cohort 2, students in the GEAR UP group have a 9% increase. For the 
PLAN science benchmark, the GEAR UP effect is stronger for Cohort 2: 
for Cohort 1 , students in the GEAR UP group have a 2% increase in the 
odds of meeting the PLAN science benchmark; for Cohort 2, students in 
the GEAR UP group have a 19% increase. 

College Readiness Standards 

The second research question can also be addressed by comparing the 
two groups with respect to improvements in level of College Readiness 
Standards. Tables 8 though 15 compare the groups with respect to the 
College Readiness Standards, with tables for each subject area and each 
cohort. Table 8 (Cohort 1) and Table 9 (Cohort 2) represent English, 

Tables 10 and 11 represent mathematics, Tables 12 and 13 represent 
reading, and Tables 14 and 15 represent science. These tables allow us 
to compare the GEAR UP group to the Non-GEAR UP group with respect 
to improvements in meeting the College Readiness Standards. For 
example, for GEAR UP students from Cohort 1, 2,619 did not meet the 
lowest level English standards on EXPLORE at grade 8. Of these 
students, 1 ,580 (60%) moved to the first level or higher of the English 
standards at grade 10 (PLAN). Note that EXPLORE and PLAN are on the 
same scale, so it is meaningful to compare standards at grade 8 to 
standards at grade 10. For the Non-GEAR UP group, 60% of the students 
that did not meet the EXPLORE English standards moved to the first level 
or higher of the standards at grade 10. So, in this case, the rates of 
improvement were identical for the GEAR UP and Non-GEAR UP groups. 
Across the different subject areas and two longitudinal cohorts, the results 
for the GEAR UP group are generally comparable to those for the 
Non-GEAR UP group. 
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Table 8: EXPLORE and PLAN English 
College Readiness Standards - Cohort 1 




Table 9: EXPLORE and PLAN English 
College Readiness Standards - Cohort 2 



Did not meet 
standards 


2,883 


1,684 


58 


2,385 


1,430 


60 


13-15 


1,651 


1,022 


62 


1,487 


973 


65 


16-19 


1,424 


542 


38 


1,316 


540 


41 


20-23 


704 


216 


31 


564 


166 


29 


24-27 


45 


15 


33 


39 


6 


15 



Non-GEAR UP 



Improved from 
EXPLORE to PLAN 



'EXPLORE 



Score Range for 
Each College 
Readiness 
Standard 



GEAR UP 



Improved from 
EXPLORE to PLAN 



•explore 



Table 10: EXPLORE and PLAN Mathematics 
College Readiness Standards-Cohort 1 
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Table 11: EXPLORE and PLAN Mathematics 
College Readiness Standards-Cohort 2 



Did not meet 
standards 


1,681 


1,235 


73 


1,520 


1,145 


75 


13-15 


2,506 


1,272 


51 


2,130 


1,189 


56 


16-19 


2,182 


645 


30 


1,909 


621 


33 


20-23 


281 


108 


38 


191 


94 


49 


24-27 


57 


24 


42 


41 


14 


34 



Non-GEAR UP 



Improved from 
EXPLORE to PLAN 



'EXPLORE 



Score Range for 
Each College 
Readiness 
Standard 



GEAR UP 



Improved from 
EXPLORE to PLAN 



•explore 



Table 12: EXPLORE and PLAN Reading 
College Readiness Standards-Cohort 1 




Table 13: EXPLORE and PLAN Reading 
College Readiness Standards-Cohort 2 
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Table 14: EXPLORE and PLAN Science 
College Readiness Standards-Cohort 1 



Did not meet 
standards 


556 


506 


91 


414 


369 


89 


13-15 


2,214 


1,204 


54 


2,076 


1,157 


56 


16-19 


3,027 


908 


30 


2,910 


900 


31 


20-23 


433 


84 


19 


377 


80 


21 


24-27 


40 


3 


8 


31 


5 


16 



Non-GEAR UP 



Improved from 
EXPLORE to PLAN 



'EXPLORE 



Score Range for 
Each College 
Readiness 
Standard 



GEAR UP 



Improved from 
EXPLORE to PLAN 



•explore 



Table 15: EXPLORE and PLAN Science 
College Readiness Standards-Cohort 2 



Did not meet 
standards 


577 


552 


96 


510 


490 


96 


13-15 


2,428 


1,444 


59 


2,146 


1,321 


62 


16-19 


3,181 


1,014 


32 


2,719 


960 


35 


20-23 


468 


74 


16 


372 


59 


16 


24-27 


53 


12 


23 


44 


3 


7 



Non-GEAR UP 



Improved from 
EXPLORE to PLAN 



•explore 



Score Range for 
Each College 
Readiness 
Standard 



GEAR UP 



Improved from 
EXPLORE to PLAN 



•explore 



We fit logistic regression models for improving level of College Readiness 
Standards from EXPLORE to PLAN that adjusts for the discrepancy in 
poverty level. We included group (GEAR UP versus Non-GEAR UP) and 
poverty level as predictors. This model was fit for each PLAN College 
Readiness Standard (English, mathematics, reading, and science) and for 
both cohorts. The results of these models are summarized in Table 16 for 
Cohort 1 and Table 17 for Cohort 2. 
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Table 16: Modeling Improvement in 
College Readiness Standards - Cohort 1 



Subject 

Area 


Predictor 


Odds Ratio 
(95% C.l.) 


Interpretation 


English 


GEAR UP 


1.15 

(1.06, 1.26) 


After adjusting for poverty level, students in the 
GEAR UP group have a higher probability of 
improving their level of English College 
Readiness Standard. 




School’s 
poverty level 


0.81 

(0.58, 1.14) 


Students from schools with higher poverty levels 
are slightly less likely to improve their level of 
English College Readiness Standard. 


Math 


GEAR UP 


1.01 

(0.92, 1.10) 


After adjusting for poverty level, students in 
the GEAR UP group have about the same 
probability of improving their level of Math 
College Readiness Standard. 




School’s 
poverty level 


0.54 

(0.38, 0.77) 


Students from schools with higher poverty levels 
are less likely to improve their level of Math 
College Readiness Standard. 


Reading 


GEAR UP 


1.07 

(0.98, 1.17) 


After adjusting for poverty level, students in 
the GEAR UP group have a slightly higher 
probability of improving their level of Reading 
College Readiness Standard. 




School’s 
poverty level 


0.53 

(0.38, 0.75) 


Students from schools with higher poverty levels 
are less likely to improve their level of Reading 
College Readiness Standard. 


Science 


GEAR UP 


1.04 

(0.95, 1.13) 


After adjusting for poverty level, students in the 
GEAR UP group have about the same probability 
of improving their level of Science College 
Readiness Standard. 




School’s 
poverty level 


0.65 

(0.46, 0.92) 


Students from schools with higher poverty levels 
are less likely to improve their level of Science 
College Readiness Standard. 
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Table 17: Modeling Improvement in 
College Readiness Standards - Cohort 2 



Subject 

Area 


Predictor 


Odds Ratio 
(95% C.I.) 


Interpretation 


English 


GEAR UP 


1.01 

(0.92, 1.12) 


After adjusting for poverty level, students in 
the GEAR UP group have about the same 
probability of improving their level of English 
College Readiness Standard. 




School’s 
poverty level 


0.61 

(0.44, 0.86) 


Students from schools with higher poverty levels 
are less likely to improve their level of English 
College Readiness Standard. 


Math 


GEAR UP 


0.94 

(0.85, 1.03) 


After adjusting for poverty level, students in the 
GEAR UP group have a slightly lower probability 
of improving their level of Math College 
Readiness Standard. 




School’s 
poverty level 


0.48 

(0.34, 0.68) 


Students from schools with higher poverty levels 
are less likely to improve their level of Math 
College Readiness Standard. 


Reading 


GEAR UP 


0.99 

(0.90, 1.09) 


After adjusting for poverty level, students in 
the GEAR UP group have about the same 
probability of improving their level of Reading 
College Readiness Standard. 




School’s 
poverty level 


0.72 

(0.52, 1.00) 


Students from schools with higher poverty levels 
are less likely to improve their level of Reading 
College Readiness Standard. 


Science 


GEAR UP 


0.92 

(0.84, 1.02) 


After adjusting for poverty level, students in the 
GEAR UP group have slightly lower probability 
of improving their level of Science College 
Readiness Standard. 




School’s 
poverty level 


0.52 

(0.37, 0.75) 


Students from schools with higher poverty levels 
are less likely to improve their level of Science 
College Readiness Standard. 



From Tables 16 and 17 we see that students at schools with higher 
poverty levels have lower probabilities of improving their level of College 
Readiness Standard. For example, for Cohort 1 we estimated that the 
odds of increasing the level of the Reading College Readiness Standard 
decreases by 15% (0.85 = 0.53 0 - 25 ) for each 0.25 increase in the school’s 
proportion of students who are eligible for free or reduced lunch. We see 
evidence of a positive GEAR UP effect, but the findings are inconsistent 
across the two cohorts. For Cohort 1 , the odds of increasing the level of 
the English College Readiness Standard are 15% higher for students in 
the GEAR UP group; for Cohort 2, the odds are about the same. For 
Cohort 1, the odds of increasing the level of the Reading College 
Readiness Standard are slightly higher for students in the GEAR UP 
group, but the effect is not statistically significant. These results suggest 
that the improvements for the GEAR UP group are only slightly better than 
the improvements for the Non-GEAR UP group. 
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Course Taking Patterns 

Research question 3 calls for a comparison of the two groups on the 
percentage of students at grade 10 who plan to take the core high school 
curriculum. Only grade 10 was examined because high school course 
taking data are not collected through the EXPLORE program. Figure 3 
summarizes the percentages of students planning to take the core high 
school curriculum. In Cohort 1, 55% of the GEAR UP students said they 
are presently taking or plan to take the core high school curriculum. In 
comparison, 5% fewer students in the Non-GEAR UP group said they are 
presently taking or plan to take the high school core curriculum. In 
Cohort 2 an equal percentage of students (57%) from the two groups said 
they were presently taking or planned to take the core high school 
curriculum. 



Figure 3: Percentage of Students at Grade 10 
Taking the Core High School Curriculum 



j | GEAR UP □ Non-GEAR UP 

60 




Cohort 1 Cohort 2 



For each cohort, we fit a logistic regression model for planning to take the 
core high school curriculum that adjusts for the discrepancy in poverty 
level and also adjusts for initial academic achievement level. We include 
group (GEAR UP versus Non-GEAR UP), school’s poverty level, and 
EXPLORE Composite score as predictors. The results for this model are 
summarized in Table 18 for Cohort 1 and Table 19 for Cohort 2. 



Table 18: Modeling Probability of Taking 
Core High School Curriculum-Cohort 1 



Predictor 


Odds Ratio 
(95% C.l.) 


Interpretation 


GEAR UP 


1.09 

(0.98, 1.21) 


After adjusting for poverty level and initial academic 
achievement, students in the GEAR UP group are slightly 
more likely to take the core high school curriculum. 


School’s 
poverty level 


1.49 

(0.99, 2.26) 


Students from schools with higher poverty levels are slightly 
more likely to take the core high school curriculum. 


EXPLORE 

Composite 


1.21 

(1.19, 1.23) 


The probability of taking the core high school curriculum 
increases with initial academic achievement level. 
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Table 19: Modeling Probability of Taking 
Core High School Curriculum-Cohort 2 



Predictor 


Odds Ratio 
(95% C.I.) 


Interpretation 


GEAR UP 


1.00 

(0.89, 1.13) 


After adjusting for poverty level and initial academic 
achievement, students in the GEAR UP group have the 
same probability of taking the core high school curriculum. 


School’s 
poverty level 


1.79 

(1.10, 2.91) 


Students from schools with higher poverty levels are more 
likely to take the core high school curriculum. 


EXPLORE 

Composite 


1.22 

(1.20, 1.25) 


The probability of taking the core high school curriculum 
increases with initial academic achievement level. 



From Tables 18 and 19, we see that 8th grade academic achievement 
level (as measured by the EXPLORE Composite score) has a pronounced 
effect on taking the core high school curriculum. For Cohort 1, we’d 
estimate that the odds of taking the core high school curriculum increase 
21% for each one point increase in EXPLORE Composite score. 
Interestingly, we also see evidence that students from high poverty 
schools are just as likely, or even more likely, to take the core high school 
curriculum. For Cohort 1, we see that students in the GEAR UP group 
were slightly more likely to take the core high school curriculum. We’d 
estimate that the odds of taking the core curriculum are 9% higher for the 
GEAR UP group. However, for Cohort 2, the odds of taking the core 
curriculum are the same for the two groups. 

College Plans 

To address the fourth research question, we compared the two groups on 
the percentage of students planning to go to a two- or four-year college. 
Table 20 summarizes how educational plans changed from grade 8 to 
grade 10 for Cohort 1 . For example, of those in the GEAR UP group that 
had no postsecondary plans in grade 8, 36% changed their plans to 
attending a two- or four-year college and 39% changed their plans to 
entering a vocational school or job training. In comparison, of those in the 
Non-GEAR UP group that had no postsecondary plans at grade 8, 

53% changed their plans to attend college and 24% decided to enter a 
vocational school or job training. While these differences might seem 
large, they are based on small sample sizes (n = 70 for GEAR UP, n = 51 
for Non-GEAR UP). This is due to the fact that almost all students (89% in 
both groups) planned to attend a two- or four-year college at grade 8. As 
described in the Method section, we excluded from the analysis those 
students who responded “other” or “undecided” to the question about 
their postsecondary plans. From Table 21, we see that the results are 
consistent for Cohort 2. 
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Table 20: Changes in Educational Plans 
from Grade 8 to Grade 10-Cohort 1 



Self-Reported Educational Plans 
10th Grade 



Two- or Four-year 
College 



•EXPLORE 



Vocational School 
or Job Training 



N 



Self-Reported Educational 
Plans in 8th Grade 


GEAR UP 


None 


70 


25 


36 


27 


39 


Vocational school or job training 


404 


189 


47 


194 


48 


Two- or four-year college 


3,805 


3,527 


93 


245 


6 


Total 


4,279 


3,741 


87 


466 


ii 


Self-Reported Educational 
Plans in 8th Grade 


Non-GEAR UP 


None 


51 


27 


53 


12 


24 


Vocational school or job training 


358 


164 


46 


180 


50 


Two- or four-year college 


3,399 


3,117 


92 


256 


8 


Total 


3,808 


3,308 


87 


448 


12 



Table 21: Changes in Educational Plans 
from Grade 8 to Grade 10-Cohort 2 



Self-Reported Educational Plans 
10th Grade 



'EXPLORE 



Self-Reported Educational 
Plans in 8th Grade 



Two- or Four-year 
College 



GEAR UP 



Vocational School 
or Job Training 



N 



None 


69 


34 


49 


18 


26 


Vocational school or job training 


422 


208 


49 


188 


45 


Two- or four-year college 


4,081 


3,744 


92 


275 


7 


Total 


4,572 


4,016 


88 


481 


11 


Self-Reported Educational 
Plans in 8th Grade 


Non-GEAR UP 


None 


44 


21 


48 


14 


32 


Vocational school or job training 


369 


181 


49 


177 


48 


Two- or four-year college 


3,540 


3,246 


92 


253 


7 


Total 


3,953 


3,448 


87 


444 


9 
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For each cohort, we fit a logistic regression model for having college 
plans in 10th grade that adjusts for the discrepancy in poverty level and 
also adjusts for initial college plans (8th grade). We include group 
(GEAR UP versus Non-GEAR UP), school’s poverty level, and 8th grade 
college plans (=1 if student planned on two- or four-year, =0 if student 
had other plans) as predictors. The results of these models are 
summarized in Tables 22 and 23. 



Table 22: Modeling Probability of 
College Plans at Grade 10-Cohort 1 



Predictor 


Odds Ratio 
(95% C.I.) 


Interpretation 


GEAR UP 


1.14 

(0.96, 1.36) 


After adjusting for poverty level and initial college plans, 
students in the GEAR UP group were slightly more likely to 
have plans for college at grade 10. 


School’s 
poverty level 


1.18 

(0.64, 2.17) 


Students from schools with higher poverty levels have 
about the same probability of having plans for college at 
grade 10. 


EXPLORE 
college plans 


14.22 

(11.79, 17.17) 


The probability of having plans for college at grade 10 is 
much higher for students with plans for college at grade 8. 



Table 23: Modeling Probability of 
College Plans at Grade 10-Cohort 2 



Predictor 


Odds Ratio 
(95% C.l.) 


Interpretation 


GEAR UP 


1.01 

(0.84, 1.22) 


After adjusting for poverty level and initial college plans, 
students in the GEAR UP group have about the same 
probability of having plans for college at grade 10. 


School’s 
poverty level 


1.88 

(1.00, 3.56) 


Students from schools with higher poverty levels have 
slightly higher probability of having plans for college at 
grade 10. 


EXPLORE 
college plans 


10.74 

(8.92, 12.94) 


The probability of having plans for college at grade 10 is 
much higher for students with plans for college at grade 8. 



From Tables 22 and 23, we see that having college plans in 8th grade 
is the major predictor of whether a student will have college plans in 
10th grade. We also see that school’s poverty level does not appear to 
affect college plans as much as it affects academic achievement. For 
Cohort 2, there is some evidence that students at schools with higher 
poverty levels have slightly higher probability of having college plans at 
grade 10, after controlling for initial (8th grade) college plans. For 
Cohort 1, the GEAR UP group was slightly more likely to have college 
plans. We’d estimate that the odds of having college plans at grade 10 
are 14% higher for students in the GEAR UP group. For Cohort 2, the 
two groups were equally likely. 
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Discussion 



In this report, we compared changes in academic readiness and college 
intent for a sample of students from GEAR UP schools to a comparable 
sample from Non-GEAR UP schools. We utilized assessment data from 
ACT’s EXPLORE and PLAN programs to measure students’ academic 
readiness and college intent at grade 8 and grade 10. Using aggregated 
EXPLORE and PLAN data and other school-level characteristics, we 
selected a sample of schools and students that were comparable to the 
GEAR UP group. By doing so, we could attribute group differences 
(GEAR UP versus Non-GEAR UP) to the intervention programs rather than 
differences in school environment. 

The outcome variables we considered include changes in overall 
academic achievement, meeting College Readiness Benchmarks and 
Standards in different subject areas, taking the core high school 
curriculum, and having plans for college. In general, if the GEAR UP 
programs had effects on these outcomes between 8th and 10th grade, 
our analyses should have shown the positive GEAR UP effects. Our 
analyses did suggest positive GEAR UP effects, though the effect sizes 
were generally small and the significant results were not consistent for the 
two cohorts studied. 

Study Limitations 

The relatively small, positive findings for the GEAR UP program are 
underestimated due to limitations with the research design. Further, the 
true test of GEAR UP’s success will come after the students from 
GEAR UP schools leave high school. GEAR UP could then be evaluated 
with respect to college enrollment, retention, and degree completion rates. 

One major limitation of this study was that we did not know which 
students at GEAR UP schools received which (if any) interventions, nor 
did we know the intensity of each student’s intervention. If only a handful 
of students at certain schools are participating in the programs, we would 
be diluting the effect of the program by analyzing data for the entire 
school. Alternatively, if our data set included measures of intervention 
intensity for each student, we would have a better opportunity to observe 
positive GEAR UP effects. 

Another limitation with our study is that we defined group membership 
(GEAR UP versus Non-GEAR UP) based on the school that the student 
attended while in 8th grade. Since students may move from school to 
school, there is no guarantee that they were enrolled at a GEAR UP 
school for an extended period of time. Also, some students will move on 
to high schools with very different academic environments. In our 
analyses, these issues will dilute any effect of the GEAR UP program. 
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The comparison sample of Non-GEAR UP schools was selected based on 
similarity to the GEAR UP schools. However, since GEAR UP schools have 
extremely high poverty levels, it is difficult to find a comparison sample 
without a discrepancy in poverty level. So, even though the Non-GEAR UP 
schools were similar to the GEAR UP schools, the comparison was not 
quite fair. To be included in our study sample, each student had to have 
taken EXPLORE in 8th grade and PLAN in 10th grade. The fact that the 
Non-GEAR UP schools participated in EXPLORE and PLAN suggests that 
these schools are also taking steps to improve the college readiness of 
their students. Therefore, we may be comparing the GEAR UP schools to 
a set of schools that also have special programs in place to help their 
students achieve college readiness. These issues will also dilute the effect 
of the GEAR UP program. 

Recommendations 

Based on our analysis and the study limitations discussed above, we 
have some suggestions for evaluating GEAR UP programs. Below, we list 
each suggestion and follow with greater discussion of each suggestion. 

▼ Tailor the analysis to the intervention. 

▼ Follow students across time. 

▼ Track students’ participation level in GEAR UP programs. 

▼ Use a control group. 

Tailor the analysis to the intervention. For example, if the goal of the 
program is to educate students about the college admissions process, 
then meaningful outcomes might be taking college prep courses, having 
college plans, and taking a standardized admissions test at the 
appropriate time. For programs that target specific academic skills 
(e.g., extra help with reading), achievement test scores may be the most 
appropriate outcome. Generally, the most appropriate outcomes will vary 
by GEAR UP program. 

Follow students across time. This allows students, and groups of students, 
to show that they are indeed improving and allows students to serve as 
their own baseline. Using a longitudinal assessment system, such as 
EXPLORE and PLAN, is valuable for several reasons, including 
1) baseline and follow-up measures are available so that changes in 
outcomes attributable to GEAR UP can be assessed; 2) the EXPLORE 
and PLAN assessments measure the same things (but at different time 
points), allowing for meaningful comparisons over time; 3) the EXPLORE 
and PLAN assessments have reliable measures of academic 
achievement, but also contain measures of students’ educational plans; 

4) the data include a wide variety of background factors, such as parents’ 
educational level and race/ethnicity, that should be controlled for in 



24 



analyses or used for subgroup analysis; and 5) comparisons can be 
made at the student level or at the school level (by aggregating 
student-level data). 

ACT’s EPAS (Educational Planning and Assessment System) consists of 
EXPLORE, PLAN, and the ACT, which students typically take in 1 1th or 
12th grade. In this study, we did not consider the ACT data because the 
cohorts have not all reached 11th or 12th grade. Cohort 1, 8th graders in 
2002-2003, will likely take the ACT during the 2005-2006 and 2006-2007 
academic years; Cohort 2, 8th graders in 2003-2004, will likely take the 
ACT during the 2006-2007 and 2007-2008 academic years. Therefore, 
when the ACT data are available, this study could be extended by also 
considering the ACT data for both cohorts. The ACT data are especially 
valuable components of EPAS because the data include students’ college 
preferences, as well as final measures of academic achievement. 

Track students’ participation level in GEAR UP programs as well as 
long-term outcomes. One of the drawbacks of the current analysis is that 
we did not know the level of participation for each student, nor did we 
know the type of intervention each student received. With this data, the 
effects of the GEAR UP programs can be better isolated and the 
evaluation made more meaningful. As discussed earlier, the true test of 
GEAR UP’s value occurs when students leave high school and have the 
opportunity to enroll in college. If possible, GEAR UP evaluators should 
track long-term outcomes, including college enrollment, retention, and 
degree completion, for the students that attended GEAR UP schools. 

Use a control group. Comparing outcomes for students from GEAR UP 
schools to a control group is an attractive study design, as long as the 
control group is similar with respect to the other factors that affect 
students’ college readiness. Possible control groups include: 

▼ The school itself. By comparing outcomes for students prior to the 
establishment of a GEAR UP program to those who come after, the 
GEAR UP effects can be measured. The strength of this approach is 
that school-level differences are naturally eliminated, so long as the 
school doesn’t undergo extensive changes during the study period. 
The drawbacks of this approach are that data must be collected for 
several years and that GEAR UP effects could be confounded with 
other changes that occur over time (i.e., other cohort effects). 

▼ A similar school. By matching on a set of relevant variables, a similar 
school or schools can be selected for comparison. While this might be 
difficult for an individual school, ACT’s EPAS program provides a rich 
source of data across thousands of schools. We used this approach in 
this study and were satisfied with the quality of the matching. The 
drawbacks of this approach were discussed earlier in the study 
limitations. 



25 



References 

ACT. (1999). PLAN technical manual. Iowa City, IA: Author. 

ACT. (2001). EXPLORE technical manual. Iowa City, IA: Author. 

ACT. (2005). Courses count: Preparing students for postsecondary 
success. Iowa City, IA: Author. 

Adelman, C. (1999). Answers in the toolbox: Academic intensity, 

attendance patterns, and bachelor’s degree attainment. Washington, 
DC: U.S. Department of Education, Office of Educational Research 
and Improvement. 

Agresti, A. (1990). Categorical Data Analysis. New York: Wiley. 

Allen, J. & Sconing, J. (2005). Using ACT Assessment scores to set 
benchmarks for college readiness. (ACT Research Report Series 
2005-3). Iowa City, IA: Author. 

Policy and Program Studies Service. (2003). National Evaluation of 
GEAR UP: A Summary of the First Two Years. Washington, DC: 

U.S. Department of Education, Office of the Under Secretary. 

Raudenbush, S.W. & Bryk, A.S. (2002). Hierarchical linear models: 
Applications and data analysis methods (2nd Ed.). Thousand Oaks, 
CA: Sage. 

Sable, J., Thomas, J., and Shen, Q. (2006). Documentation to the NCES 
Common Core of Data Public Elementary/Secondary School Universe 
Survey: School Year 2003-04, (NCES 2006-324). U.S. Department of 
Education. Washington, DC: National Center for Education Statistics. 

SAS Institute, Inc. 2004. SAS/STAT® 9. 1 User’s Guide. Cary, NC: SAS 
Institute, Inc. 

Terenzini, P.T., Cabrera, A.F., Deil-Amen, R., Lambert, A. (2005). The 
Dream Deferred: Increasing the College Preparedness of At- Risk 
Students. Washington, DC: U.S. Department of Education. 

Wolfinger, R., and O’Connell, M. (1993). Generalized linear mixed models: 
A pseudo-likelihood approach. Journal of Statistical Computation and 
Simulation, 48, 233-243. 



27 



Appendix 

Table 24: School-level Characteristics for Cohorts Studied 



School-level Characteristics 


Cohort 1 
(N = 119) 


Cohort 2 
(N = 136) 


GEAR UP 


Non- 

GEAR UP 


GEAR UP 


Non- 

GEAR UP 


Public (%) 


100.0 


100.0 


100.0 


100.0 


Enrollment Size (%) 










1-99 


1.7 


2.5 


1.5 


2.2 


100-199 


14.3 


15.1 


10.3 


13.2 


200-299 


18.5 


23.5 


17.7 


23.5 


300-499 


26.9 


26.9 


32.4 


33.8 


500-999 


35.3 


30.3 


33.1 


25.0 


1,000-2,499 


3.4 


1.7 


5.2 


2.2 


Metropolitan Area (%) 










Urban 


16.0 


16.0 


14.7 


14.7 


Suburban 


1.7 


1.7 


2.2 


2.2 


Rural 


82.4 


82.4 


83.1 


83.1 


Poverty Level (%) 










5-1 1 .9 


9.2 


21.9 


7.4 


20.6 


12-24.9 


61.3 


54.6 


57.4 


61.8 


25 or more 


29.4 


23.5 


35.3 


17.7 


Mean number of EXPLORE 
tested students 3 


52.7 (50.3) 


49.8 (46.0) 


49.3 (46.4) 


45.7 (39.6) 


Mean EXPLORE Composite 


14.5 (1.4) 


14.5 (1.3) 


14.3 (1.3) 


14.3 (1.3) 



Percentages may not add to 100 due to rounding or missing responses. 
a AII EXPLORE tested students were in 8th grade. Numbers in parentheses are standard 
deviations. 
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Table 25: Student-level Characteristics for Cohorts Studied 





Cohort 1 


Cohort 2 






Non- 




Non- 


Student-level Characteristics 


GEAR UP 


GEAR UP 


GEAR UP 


GEAR UP 


Total number of students 


6,270 


5,808 


6,707 


5,791 


Female (%) 


53.5 


52.7 


54.4 


53.2 


Race/Ethnicity (%) 

White 


58.1 


60.7 


57.6 


66.5 


African American 


13.6 


20.5 


17.9 


19.3 


Hispanic 


11.7 


4.0 


8.3 


3.0 


Mother’s Educational Attainment (%) 

Did not complete high school 


15.3 


11.9 


14.3 


11.5 


Have a high school diploma or 
equivalent 


30.0 


30.0 


28.4 


29.2 


Career/technical training 


3.8 


3.9 


3.6 


3.8 


Some college no degree 


11.2 


10.8 


11.0 


11.0 


2-year college degree 


8.3 


6.9 


8.4 


8.1 


4-year college degree 


10.9 


10.1 


10.1 


12.3 


More than 4 years of college 


5.1 


5.4 


5.5 


5.2 


Father’s Educational Attainment (%) 

Did not complete high school 


16.1 


12.8 


15.8 


12.1 


Have a high school diploma or 
equivalent 


28.1 


26.1 


27.1 


26.4 


Career/technical training 


8.6 


9.4 


7.6 


9.3 


Some college no degree 


7.3 


7.1 


6.5 


7.3 


2-year college degree 


3.7 


3.7 


4.1 


4.5 


4-year college degree 


8.2 


7.4 


7.4 


8.8 


More than 4 years of college 


4.8 


4.2 


4.2 


4.4 



Percentages may not add to 100 due to rounding or missing responses. 
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